BIE State Systemic Improvement Plan 
Evaluation Plan 


EVALUATION DESIGN 


In accordance with the Office of Special Education Programs (OSEP) Guidance and Review 
Tool, the evaluation plan will be aligned to the BIE Theory of Action’s three strands (e.g., 
marketing the value of education, professional development, and Data-Based Decision Making) 
and the logic model input and outputs. Formative and summative evaluation questions focus on 
short, medium, and long-term outcomes that will inform the measurement of SSIP 
implementation and impact of achieving BIE’s State-Identified Measurable Result of increasing 
the number of youth engaged (e.g., enrolled in higher education, competitively employed) one 
year out of high school. A critical component of the evaluation plan will include the 
dissemination of information to BIE SSIP stakeholders for their input as the DPA examines the 
effectiveness of the implementation, assessment of progress toward achieving intended 
improvements, and making modifications to the SSIP as appropriate. This evaluation plan will 
be conducted internally by DPA staff in collaboration with the ADD Regions and BIE offices 
being proposed in the new reorganization (e.g., Assessment and Accountability Offices, 
Academic Offices) [Component 3(a)]. This section provides information on the design of the 
evaluation. Information includes evaluation questions, stakeholder information needs emerging 
from the evaluation, and the evaluation design. 


Formative Evaluation Questions: 
Marketing the Value of Education 


1. Will initiating and participating in relevant interagency activities build sufficient and 
effective networks across high schools for information sharing and support? 


2. Will participation in interagency activities help DPA to identify supports and services for 
youth with disabilities across high schools? 


Professional Development 


3. How effective is our system of determining high schools in need of technical assistance and 
professional development to support secondary transition? 


4. Has the DPA provided sufficient support to high schools in order for high schools to 
establish networks and provide TA/PD at the local level? 


Data-based Decision Making Process 


5. How are high schools using fiscal data to develop or refine transition programs focused on 
improving post-school outcomes for youth with disabilities? 


6. How are high schools using funds to engage families and community members in providing 


support to high schools? 


7. How are high schools using DBDM tools and resources provided to identify current 
secondary transition needs? 


8. How many high schools are accessing data via STEPSS and NASIS for the purposes of 
improving secondary transition programs? 


9. How many high schools are reviewing secondary transition data from year-to-year and 
identifying trends? 


Summative Evaluation Questions: 
Short-term: 


10.Is there an improved understanding of the importance of education among tribal community 
members? [Marketing the Value of Education] 


11.Was there increase in communication and collaboration among DPA, ADD Regions, high 
schools and communities? [Marketing the Value of Education, Professional Development] 


12.Did high schools increase their knowledge of data-based decision making? [DBDM, 
Professional Development] 


Medium-term: 


13.Are there an increased number of students and parents engaging in transition planning and 
instruction? [Marketing the Value of Education] 


14.Was there an increase in the number of high schools using data to determine EBPS to 
implement in high schools? /DBDM] 


15.Was there an increased number of teachers implementing evidence-based strategies to teach 
secondary transition skills (e.g., job skills)? [DBDM, Professional Development] 


16.Was there an increased number of youth receiving appropriate secondary transition services 
and supports (SPP/APR Indicator 13)? [Marketing the Value of Education , DBDM, 
Professional Development] 


17.Was there increased academic achievement of youth? [Marketing the Value of Education, 
DBDM, Professional Development] 


18.Was there an increased number of youth with skills and knowledge to get a job or go to 
postsecondary education or training? [Marketing the Value of Education , DBDM, 


Professional Development] 


19.Was there an increased number of youth graduating with a high school diploma or equivalent 


(SPP/APR Indicator 1, 2)? [Marketing the Value of Education , DBDM, Professional 
Development] 


Long-term: 


20.Was there an increase in the number of youth enrolled in higher education, competitively 
employed, enrolled in postsecondary education or training, or other employment (SIMR; 
SPP/APR Indicator 14)? [Marketing the Value of Education , DBDM, Professional 
Development] 


Dissemination and Use of Evaluation Data 


DPA, in collaboration with ADD Regions and primary stakeholder group, will collect and 
summarize the evaluation data and disseminate to larger stakeholder group (primary & secondary 
stakeholders) for analysis and decisions regarding effectiveness of the implementation of the 
SSIP and the progress towards achieving positive post-school outcomes for youth with 
disabilities in BIE-funded high schools. Data collected from evaluation will inform program 
improvement across BIE-funded high schools in the three ADD Regions [Component 3(d)]. 


DPA will be responsible for ensuring all data collectors are properly trained to collect the data 
(e.g., post-school outcome survey, focus groups) via existing infrastructures for data collection, 
Analysis, and use. DPA will rely on the support of the BIE data team and the national OSEP 
funded TA providers to ensure BIE has valid and reliable data to disseminate to high schools for 
use. DPA has refined the NASIS special education data collection procedures and continue to 
make improvements to the existing mechanisms to improve the reliability and validity of data 
collected for the purposes of the SPP/APR and SSIP. Written procedures for data collection, 
analysis, and use will be made available to stakeholders and updated annually. DPA will train the 
ERCs (once established) on the procedures for data collection, analysis, and use of data for SSIP 
evaluation. The ERCs will then be responsible for outreach to high schools to ensure the 
procedures are implemented with fidelity. All data collection activities, procedures, and 
processes will be published for stakeholder review on the special education section of the BIE 
website. High Schools will be responsible for disseminating information regarding data 
collection, analysis, and use to families in their communities (e.g., data night, meeting to discuss 
procedural safeguards, annual performance report) [Component 3(b)]. 


Evaluation Design 


The DPA will be conducting a non-experimental longitudinal design for the evaluation. Data will 
be collected from multiple measures at multiple points in time. Change and trends will be 
examined over time. This method was determined based on the nature of the evaluation 
questions and the DPA’s current capacity to carry out the SSIP evaluation. 


